Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses critical test gaps in the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request improves test coverage for the verifier report helpers, specifically for summary generation and file output logic. The changes are a good step forward. I've identified an opportunity to make the new summary preference test more comprehensive and maintainable by using a data-driven approach with it.each and adding missing test cases. This will ensure all summary logic branches are robustly covered.
| it("prefers the highest-severity summary branch", () => { | ||
| expect( | ||
| buildVerifyReportOutput({ | ||
| clarified: makeReport("semantically clarified but not fully proven"), | ||
| }).summary, | ||
| ).toBe("semantically clarified but not fully proven"); | ||
|
|
||
| expect( | ||
| buildVerifyReportOutput({ | ||
| proven: makeReport("proven working"), | ||
| clarified: makeReport("semantically clarified but not fully proven"), | ||
| blocked: makeReport("blocked by setup/state"), | ||
| }).summary, | ||
| ).toBe("blocked by setup/state"); | ||
|
|
||
| expect( | ||
| buildVerifyReportOutput({ | ||
| proven: makeReport("proven working"), | ||
| deeper: makeReport("deeper issue remains"), | ||
| blocked: makeReport("blocked by setup/state"), | ||
| }).summary, | ||
| ).toBe("deeper issues remain"); | ||
| }); |
There was a problem hiding this comment.
While this test covers several important cases for summary preference, it could be more comprehensive and maintainable. Specifically, it's missing a test case where proven working is the expected summary, and it doesn't explicitly test the precedence of semantically clarified... over proven working as described in the PR description.
I suggest refactoring this into a data-driven test using vitest's it.each. This makes the different scenarios clearer, improves readability, and ensures all levels of severity precedence are explicitly tested.
it.each([
[
"only proven reports are present",
{ proven: makeReport("proven working") },
"proven working",
],
[
"clarified reports are present with proven reports",
{
proven: makeReport("proven working"),
clarified: makeReport("semantically clarified but not fully proven"),
},
"semantically clarified but not fully proven",
],
[
"blocked reports are present with clarified reports",
{
clarified: makeReport("semantically clarified but not fully proven"),
blocked: makeReport("blocked by setup/state"),
},
"blocked by setup/state",
],
[
"deeper issue reports are present with blocked reports",
{
deeper: makeReport("deeper issue remains"),
blocked: makeReport("blocked by setup/state"),
},
"deeper issues remain",
],
])("prefers the highest-severity summary branch when %s", (_caseName, reports, expectedSummary) => {
expect(buildVerifyReportOutput(reports).summary).toBe(expectedSummary);
});1 participant
Issue
The verifier-output refactor introduced new helper branches in
scripts/verify-report.ts, but the initial test only covered one happy-path summary and alias calculation case. That left the new summary-severity ordering and JSON file-write behavior effectively unguarded.Root cause
The recent extraction focused on moving output shaping and persistence into reusable helpers for
verify-layer1-remaining.ts, but the follow-up test only asserted aggregate counts for a mixed report fixture. The branches that decide which summary string wins and whether--outputactually writes a file were not exercised.Fix
This change keeps scope tight to the new helper and adds focused tests in
scripts/verify-report.test.tsfor the missing paths. The new coverage checks that semantically clarified reports win over proven-only output, that blocked reports outrank clarified output, that deeper issues outrank blocked output, and thatwriteVerifyReportOutputis a no-op without a path while still writing newline-terminated JSON when a path is provided.Validation
pnpm vitest run scripts/verify-report.test.ts